1. Field of the Invention
The present invention relates to a user interface feedback apparatus, a user interface feedback method, and a program therefor. More particularly, the present invention relates to a user interface feedback apparatus, a user interface feedback method, and a program able to provide user interface feedback in the form of a tactile response when the user operates a switch or similar operable element displayed on a touch panel, for example.
2. Description of the Related Art
Currently, user interfaces are being used in a variety of fields wherein switches or similar operable elements are displayed on a liquid-crystal display or other type of touch panel, the switches being operable by the user's fingertips. Such touch panel interfaces are configured to display switches, buttons, sliders, dials, or other elements as images, and subsequently execute predetermined processing when contact with the user's fingertip is detected with respect to a particular screen display region.
However, when operating such an interface, the user does not receive the same tactile response experienced when physically moving an actual switch or slider. To address this issue, there have been proposed interfaces that provide user interface feedback by producing resistance, vibration, or a sound replicating the response of an actual switch. Such interfaces are referred to as tactile interfaces, and are described in Japanese Unexamined Patent Application Publication Nos. 2003-330611, 2004-21528, and 2003-288158, for example.
As an example, one tactile interfaces involves installing piezoelectric actuators around a touch panel. When the user presses an operable button on the touch panel, vibrations and sounds are produced, thereby providing user interface feedback similar to that of physically operating an actual button.
The following three categories exemplify the types of tactile interfaces that provide user interface feedback.
(1) Vibrotactile devices
(2) Two-dimensional (2D) tactile displays and matrices
(3) Electrical stimulation devices
The above interface types are summarized below.
(1) Vibrotactile Devices
Interface devices that produce vibrations or similar effects in response to user operations are referred to as vibrotactile devices. Vibrotactile devices are configured to produce vibrations corresponding to contact with the user's fingertips, for example, with the use of actuators or similar elements that convert an electrical signal into vibration.
A torque-based tactile feedback device is one example of a vibrotactile device. A torque-based tactile feedback device creates feedback from the torque produced when starting or stopping the rotation of a motor, which is then felt by the user as user interface feedback. (See for example: M. Yoshie, H. Yano, H. Iwata, “Development of non-grounded force display using gyro moments,” Proceedings of Human Interface Society Meeting, 2001, pp. 25-30; and Y. Fukui, S. Nishihara, K. Nakamura, J. Yamashita, “Hand-held torque feedback display,” Proceedings of SIGGRAPH01 Abstract and Applications, 2001, ACM, p. 192.)
However, with such a configuration, the production of sufficient torque involves incorporating a motor of comparable size, which is difficult to install in a compact portable device. Moreover, a configuration using such a motor is only able to provide feedback following specific patterns, and might not be able to produce arbitrary patterns. It is also difficult to suddenly stop the output of feedback from the device once started.
Many of the vibration motors used in devices such as mobile phones and game controllers are provided with asymmetric axes or cams. Vibration is thus initiated from such a configuration after a fixed delay time. Since the actuators exhibit a considerably large latency, the above configuration is ill-suited to interactive applications.
In addition, piezo actuators, voice coils, and magnetic actuators such as solenoids are also used as elements for vibrating the entire device and directly stimulating the user's hand, for example. (See for example: R. Hughes and A. Forrest, “Perceptualisation using a tactile mouse,” Visualization '96, IEEE, 1996; M. Fukumoto and S. Toshiaki, “ActiveClick: Tactile feedback for touch panels,” CHI'2001, Extended Abstracts, ACM, 2001; I. Poupyrev, S. Maruyama et al., “Ambient Touch: Designing tactile interfaces for handheld devices,” UIST'2002, ACM, 2002; and I. Poupyrev and S. Maruyama, “Tactile interfaces for small touch screens,” UIST, ACM, 2003.)
(2) Two-Dimensional (2D) Tactile Displays and Matrices
Currently there exist tactile displays premised on a matrix provided with a plurality of pins. The user's hand is then affected by the action of the pins. There are two types of such displays. The first type forms a three-dimensional relief by moving the pins in the vertical direction (see for example: S. Frisken-Gibson, P. Bach-y-Rita et al., “A 64-solenoid, four-level fingertip search display for the blind,” IEEE Transactions on Biomedical Engineering BME-34(12): 963-965, 1987; M. Shinohara, Y. Shimizu et al., “Three-dimensional tactile display for the blind,” IEEE Transactions on Rehabilitation Engineering 6(3): 249-255, 1998; and I. Poupyrev, T. Nashida et al., “Actuation and Tangible User Interfaces: the Vaucanson Duck, Robots, and Shape Displays,” Tangible and Embedded Interaction, ACM: 205-212, 2007). The second type causes the pins to vibrate in both the horizontal and the vertical directions (see, for example, R. Cholewiak and C. Sherrick, “A computer-controlled matrix system for presentation to skin of complex spatiotemporal patterns,” Behavior Research Methods and Instrumentation 13(5): 667-673, 1981). However, the above configurations are also comparatively large in size, and it is difficult to incorporate such configurations into compact portable devices.
(3) Electrical Stimulation Devices
Electrical stimulation devices have also been proposed, which transmit user interface feedback by causing a weak electric current to flow into the user's hand, for example. For example, such a device may be configured to stimulate the user by means of electrodes arranged in a matrix pattern (see for example: H. Kajimoto, N. Kawakami et al., “Electrocutaneous display as an interface to a virtual tactile world,” VR'2001, IEEE, 2001; and K. Kaczmarek and P. Bach-y-Rita, “Tactile displays,” Virtual Environments and Advanced Interface Design, ed. by W. Barfield and T. A. Furness, Oxford Univ. Press, Oxford: 349-414, 1995). However, the configuration of such electrical stimulation devices is such that, for many users, the electrical stimulation feels uncomfortable. Furthermore, there is a problem in that the discomfort remains even after removing the current.
[Electrovibrotactile Displays]
Electrovibrotactile displays will now be described. An electrovibrotactile display is a device that utilizes a skin-attracting effect whereby the skin of a person's hand or fingertips is attracted as a result of applying intermittent voltage to an electrically insulated surface. This physical phenomenon was unexpectedly discovered in 1953 by Mallinckrodt, Hughes et al. (see, for example, E. Mallinckrodt, A. Hughes et al., “Perception by the skin of electrically induced vibrations,” Science 118(3062): 277-278, 1953). Mallinckrodt et al. discovered that if a voltage of 110 V is intermittently applied to a metal and a dry fingertip is dragged across the surface of the metal covered with an insulating layer, a rubbery feeling is produced.
The principle behind this physical phenomenon was analyzed by Mallinckrodt et al. as follows.
(1) The insulating layer of the dry skin acts as the dielectric material of a condenser with the metal as one capacitor, while the water (i.e. conducting material) contained in the finger constitutes the condenser and acts as another capacitor.
(2) When an alternating voltage is applied to the condenser, intermittent attractive force is produced between the skin and the metal.
(3) If the hand is moved while lightly pressing the surface, the friction between the skin and the plate surface periodically varies in magnitude, and the operator experiences the sensation of his or her finger being repeatedly attracted and released.
Although this phenomenon was discovered in 1953, practical use was not realized until 1970. In 1970, R. M. Strong developed a tactile display based on the above principle (see, for example, R. M. Strong and D. E. Troxel, “An electrotactile display,” IEEE Transactions on Man-Machine Systems 11(1): 72-79, 1970). Strong proposed a display made up of an array of many pins covered with a dielectric material. If an intermittent voltage is applied to the pins and a finger is then moved over the pins, a tactile response is felt. More specifically, a distinct sensation of the finger being pulled is felt.
However, since the fusion of input and sensing technologies was insufficiently realized in proposed tactile displays at the time, the utility value of Strong's proposal was limited. Since the proposed configuration involved the use of a large number of pins, its use in a typical flat display, for example, was unfeasible.
In 1998, a similar tactile display was proposed by Tang and Beebe (see, for example, H. Tang and D. Beebe, “A microfabricated electrostatic haptic display for persons with visual impairments,” IEEE Transactions on Rehabilitation Engineering 6(3): 241-248, 1998). As a result, a thin, durable display was realized by means of microfabrication technology using applied lithography. The underlying principle, however, was identical to Strong's.
Additionally, an electrovibrotactile display different from the above was proposed by Yamamoto, Nagasawa et al. in 2006 (see, for example, A. Yamamoto, S. Nagasawa et al., “Electrostatic tactile display with thin film slider and its application to tactile telepresentation systems,” IEEE Transactions on Visualization and Computer Graphics 12(2): 168-177, 2006). The proposed display is configured as shown in FIG. 1, wherein a finger 31 moves a slider 20 placed upon a plate electrode 11 having the electrode pattern shown in the figure.
The slider 20 includes an aluminum layer 21 and an insulating layer 22. An electrode pattern is formed inside the plate electrode 11 such that + and − voltages are applied in alternation. A periodically varying voltage signal, like that shown by the applied voltage pattern 40 in FIG. 1, is applied to the electrodes within the plate electrode 11. As a result of this process, electrostatic force is produced between the conductive material of the slider 20 and the plate electrode 11. Note that in this configuration, the finger 31 only contacts the insulating layer 22, and thus a current is not produced. However, the electrode patterning in this configuration makes it difficult to implement in the displays of small devices.